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Abstract 

This article is a contribution to the domain of (convergent) deforma- 
tion quantization of symmetric spaces by use of Lie groups representation 
theory. We realize the regular representation of SL(2, R) on the space of 
smooth functions on the Poincare disc as a sub-representation of SL(2, R) 
in the Weyl-Moyal star product algebra on R 2 . We indicate how it is 
possible to extend our construction to the general case of a Hermitian 
symmetric space of tube type. 
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1 Introduction 

For the convenience of the reader, we start by recalling basic facts in deformation 
quantization theory which will be useful in explaining what is done in the present 
article. 

1.1 Star products 

We first recall the notion of deformation quantization (star product) as intro- 
duced in the seminal paper ||. Star products were originally introduced as an 
autonomous formulation of quantum mechanics in the framework of classical 
mechanics, that is without reference to a Hilbert space representation. Roughly 
speaking, if one thinks of a mechanical system as the data of a symplectic 
manifold M (the phase space) together with some fixed function H on M (the 
hamiltonian) , one usually understands a quantization of the classical system 
(M, H) as a correspondence (the quantization map) between functions on M 
(the classical observables) and operators (the quantum observables) on some 
Hilbert space (the quantum states of the system) . The physical information is 
encoded in the spectral theory of the algebra of quantum operators. In partic- 
ular, by reading the composition product of quantum operators at the level of 
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classical functions on M via the quantization map, one should get an associative 
product on the classical functions encoding the quantum mechanics. Typically, 
a star product on a symplectic manifold M can be thought as such a transported 
composition product on functions on M. It turns out that star products can be 
constructed independently of any Hilbert space data. Now precisely, one has 

Definition 1.1 Let (M,u>) be a symplectic manifold and E = C°°(M) be the 
commutative algebra of smooth (complex valued) functions on M. A formal 
deformation quantization (or star product) of (M,u>) is an associative product 
law on the space E[[v]] of formal power series in the parameter v with coefficients 
in E. Denoting this product by 

(a, 6) i-> a* u b, a,b & E[[v]]. (1) 

one requires furthermore the following properties. 

(i) The map *„ is <C[[v]]-bilinear (C[[v]] stands for the formal power series 
with constant complex coefficients). 

(ii) If for u,v G C°°(M) C -E[hv]], one denotes 

oo 

u* v v =: v k c k (u, v), (2) 

k=0 

then the maps Ck : E — > E are bidifferential operators on E such that 
Co(u,v) = uv and c\{u,v) — {u, v} where {, } stands for the Poisson 
bracket on E associated with the symplectic structure ui. 

In other words, a formal deformation quantization of a symplectic (or more 
generally Poisson) manifold is a formal associative deformation of the usual 
commutative product on functions in the direction of the Poisson structure. 

Definition 1.2 Two star products -k^ 1 and -kj 2 on M are called equivalent if 
there exists a formal series of differential operators T = Id + Ylk>i ^Tk such 
that for all u,v € C°°(M), one has T(u* v l v) = Tu* u2 Tv. 

On every Poisson manifold there exists a star product pl| , but the most simple 
example of a star product is the so-called Weyl-Moyal star product on the 
symplectic vector space (M = M. 2n ,ui = Yli<j A-ijdx* A dx^) (the coefficients 
Ay 's being constant). The Weyl-Moyal star product on R 2n is defined as the 
C[M]-bilinear extension to C°°(M. 2n )[[v}} of 

u *° v v := uv + Y ^A ilJ1 . . . A« fc - . 9 \ . u - . ^ . v . (3) 

k=l 

It turns out to be associative. The associative algebra (C 00 (IR. 2n )[[^]],*^) is 
called the Weyl-Moyal algebra on K 2 ™ . It can be seen as a completion of the uni- 
versal enveloping algebra of the Heisenberg Lie algebra constructed on (R 2n , ui). 
It can also be characterized-and that is what matters here- by its automorphism 
group (see below). 
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1.2 Symmetries 



In presence of symmetry i.e. when one is given a symplectic action of some Lie 
group G on [M, to), one may want our quantization map to be equivariant with 
respect to this action and some unitary representation of G on the state space. 
At the level of formal deformation quantization, this amounts to require the 
G-invariance of the star product. 

Definition 1.3 Let (Af, oj) be a symplectic manifold which a Lie group G acts 
on in a differentiable way by symplectomorphisms (i.e. denoting the action by 
G x M — » M : (g,x) <— > T g {x), one has t*lu = lo Vg € G). A star product * v 
on (M,lu) is called G-invariant if for all functions u,v G C°°(M) and g G G, 
one has 

T*(U* U V) = TgU*„T*V. (4) 

When the action of G preserves a symplectic afhne connection on M, the exis- 
tence of a G-invariant star product is guaranteed and these star products are 
classified ||, 13 . In particular, up to a change of parameter v i— > v' , the Weyl- 



Moyal star product is the only star product on the symplectic vector space 
which is invariant under Sp(n, R) x M. 2n acting on R 2rl by affine transforma- 
tions with respect to the canonical flat connection. In particular, this very basic 
example already shows an interplay between deformation quantization and ge- 
ometry in the sense that a star product generally carries a strong geometrical 
information. 



1.3 Convergence 

Recovering from the data of a formal star product the spectral theory containing 
the physical information is generally a very hard problem, especially in presence 
of curvature. This problem appears to be closely related to the one of finding 
"convergent" (or "non- formal" ) star products jl3], |TJ, ^| . Many notions of con- 
vergence for star products can be found in the literature, but it's probably fair 
to say that a convergent star product is a star product which admits an asymp- 
totic representation in some operator algebra. For instance the G*-algebraic 
situation has been developed by Rieffel in |^p|| and in earlier works. However, 
the geometrical affine information encoded in the deformed product in these 
cases is always associated to a flat affine connection as in the Weyl-Moyal case. 
It is therefore challenging to try to attach 'a la Rieffel' some topological oper- 
ator algebra framework, such as a G* or Frechet, to symmetry invariant star 
products on (symplectic) symmetric spaces. For solvable symplectic symmetric 
spaces (i.e. whose transvection group is a solvable Lie group) , this has been done 
in Later, other solvable Lie group actions have been treated in the same 
spirit in . These works have led to oscillating integral formulae for symmetry 
invariant associative products on function spaces defined on solvable symmetric 
spaces. The associated oscillating kernels have been explicitly determined and it 
turned out that they entirely encode the affine symmetric symplectic geometry 
of the space at hand (topologically R 2n ). For a beautiful symplectic geometrical 
interpretation of the phase part occurring in such an oscillating kernel see p5[ . 
In a different context than star product theory, non-formal quantization of 
semisimple symmetric spaces has extensively been studied in the important 
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work of Unterberger (see [E3| for a survey) by using techniques of harmonic 
analysis (see also |Q for an operator algebraic approach). 

1.4 The question attended here and the results obtained 

In the present article, we are concerned with investigating the problem of con- 
structing invariant associative products on certain function algebras on semisim- 
plc symplcctic symmetric spaces by applying to the semisimple situation the 
same method as used for the solvable case in Q . This method essentially relies 
on the fact that formal star products on a symplectic space M are classified 
(up to equivalences) by the space of formal powers series with coefficients in the 
second de Rham cohomology space of M In particular, when the topology 
of the space is trivial all the star products are equivalent to one another. If one 
disposes of a Darboux chart on a G-space, one should then be able to relate 
by an equivalence any G-invariant star product to the simplest (non-invariant!) 
one: the Weyl-Moyal star product. For certain symmetric spaces, the above 
mentioned equivalence T can be obtained quite explicitly from a combination 
between fairly old techniques in star representation theory (Lie group repre- 
sentation theory in the framework of deformation quantization |], ^| and a 
detailed geometrical study. Non-commutative harmonic analysis really enters 
in the game when passing from this formal situation to a convergent one. 
We now precise the construction. Let G/K be an Hermitian symmetric space 
of tube type. The first step consists in defining an action by star product 
derivations of the Lie algebra g of G on the Weyl-Moyal algebra on R m . This 
representation of g has already been considered in j^] and further studied in Q . 
It is a ;/-deformation of the regular infinitesimal action of q on the functions on 
(a dense open Darboux chart domain of) the causal symmetric space of Cayley 
type G/H associated to G/K. For details, see Section 0. 

The second step, performed in the present paper, consists in finding a subspace 
of the Weyl-Moyal algebra to which the representation restricts as (equivalent 
to) the regular infinitesimal action of g on functions on the tube G/K . This 
can be achieved only when the parameter v is assigned some particular rational 
values. In this case, an injective intcrtwincr 

C™{G/K) Weyl-Moyal algebra 

is constructed via the Plancherel formula for L 2 (G/K,8) where S is some finite 
dimensional unitary representation of K depending on v. As a consequence the 
restriction of the Weyl-Moyal product to the image of the intertwiner defines 
(at least formally) a G-equivariant mapping 

Cf{G/K) x Cf(G/K) Weyl-Moyal algebra : (u,v) i-> u* v v. 

This construction yields an interpolation between the regular representations of 
G on the function on the Cayley symmetric space G/H on the first hand and on 
the corresponding tube G/K on the second hand. In particular, one is tempted 
to make the analogy between the latter and the interpolation found by Neretin 
in the compact-non-compact duality context p8|. 

We do not know wether it is possible to choose the intertwiner in a way that 
the algebra generated by the subspace C£°(G/K) in the Moyal algebra would 
still be a function algebra on G/K i.e. be part of the regular representation of 
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G on C°°(G/K) (in the solvable situation, it is actually possible ||). This last 
question certainly deserves quite a bit of interest, and we hope to be able to 
give some elements of answer to it in a future work. 

Throughout this paper, and in order to avoid technicalities we concentrate our- 
selves to the "test" case SL(2,M)/SO(2). However, we shall give concrete indi- 
cations on how to extend our construction to the general case of an arbitrary 
Hermitian symmetric spaces of tube type G/K. 
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2 Covariant ★-products 

We start by recalling some basics in Hamiltonian systems and then pass to 
star products. In this section, [M, uj) is a symplectic manifold and g is a finite 
dimensional real Lie algebra. One assumes one has a representation of g as an 
algebra of symplectic vector fields on (M,oj). That is, one has a Lie algebra 
homomorphism g — > X{M) : X i— > X* (X(M) stands for the space of smooth 
vector fields on M) such that for all X in g one has 

C x *w = 0, 

where C denotes the Lie derivative. One supposes furthermore that this repre- 
sentation of g is strongly Hamiltonian which means that there exists a R-linear 
map 

g ^ C°°{M) : X^X X , 

such that 

(i) d\ x = ix'U, 

(") \x,Y] = {^X, Ay}, 

where {, } denotes the Poisson structure on C°°(M) associated to the symplectic 
form w. 

Definition 2.1 A quadruple (M,u>, Q, X) with {M,oj),g and X as above is called 
a strongly hamiltonian system. The map A:gi-> C°° (M) is called the moment 
mapping. 

Example 1. Coadjoint orbits. Let M = O C g* be a coadjoint orbit of a Lie 
group G with Lie algebra g. In this case, we denote by g — > X(M) : IhI* 
the rule which associates to an element X in g its fundamental vector field on 
O : 

X* := j t \ Q Ad*(eM~tX))x, 

where Ad*(g)x denotes the coadjoint action of the element g G G on x £ g* . 
The formula oj®(X*,Y*) := (x, [X, Y]) (with X, Y € g) then defines a symplec- 
tic structure called after, Kirillov, Kostant and Souriau, the KKS symplectic 
form on O. Defining, for all X £ g, the function Ax € C°°(0) by 

Xx(x) := (x,X), 
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one then has that the quadruple (0,u>°,g, A) is a strongly Hamiltonian system. 
In the setting of deformation quantization there is a natural way to define the 
equivariant quantization of a classical hamiltonian system |jj , ]3| , ]l4| . 

Definition 2.2 ^ Let (M, w,0, A) be a classical strongly hamiltonian system. 
A star product *„ on (M,u>) is called g-covariant if for all X, Y £ g one has 

Xx *u Ay — Ay *„ Xx = 2v{\x, Ay}. 

In order to avoid technical difficulties in defining the star representation (see 
below) , we will assume our covariant star products to satisfy the following con- 
dition. 

Definition 2.3 Let (M,u,g, A) be a strongly hamiltonian system. Let *„ be a 
g—covariant star product on (M,w). We say that -k v has the property (B) if 
there exists an integer N € N such that one has 

(Xx * v u) mod(i/ N ) = (Xx * v u) mod(v N+n ); 
(u * u X x ) m.od(v N ) = (u * v Ax) vao&(v N +n ) , 

for all X e g and u € C°°(M) and n £ N. 

In other words the series Ax *„ u and u * v Xx stop at order N independently of 
u in C°°(M). The following definition then makes sense ||. 

Definition 2.4 Let (M,lo,q,X) be a strongly hamiltonian system and *„ be a 
Q-covariant star product on (M,u>) satisfying the property (B). One defines the 
representation ad*„ of g on E v by 

a,d^(X)a := —(Xx *v a - a* v X x )- (5) 
2v 

Definition 2.5 Let G be a connected Lie group with Lie algebra g. Let (M, oj, g, A) 
be a strongly hamiltonian system. Let *„ be a g-covariant star product on (M, u>) 
satisfying the property (B). The associated star representation of G (if it exists) 
is the representation ir of G on E v such that 

d-K = ad*^ . 

3 Holonomy reducible symmetric orbits 

Let G be a connected simple Lie group. Let us denote by g its Lie algebra. Let 
O be an adjoint orbit of G in g. Choose a base point o in O and denote by t) the 
Lie algebra of its stabilizer in g. Since g is simple the Killing form (3 establishes 
an equivariant linear isomorphism between g and its dual g*. Therefore, every 
adjoint orbit can be identified with a coadjoint one. We will denote by u> the 
KKS symplectic structure on O (cf. Example 1). 

Definition 3.1 An adjoint orbit O in g is symmetric if there exists an involu- 
tive automorphism a of g such that f) = {X £ g\a(X) = X}. In this case, we 
denote by g — [) © q the decomposition of g induced by a. 

One has [t), q] = q and [q, q] = f) and the following is classical. 
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Proposition 3.1 Let O be a symmetric adjoint orbit of a simple Lie group 
G and (fl,cr, 51) be its associated symplectic symmetric triple. The following 
assumptions are equivalent. 

(i) the center 3(f)) of f) contains a non-compact element. 

(ii) The subspace q splits into a direct sum q = t® I of isomorphic ^-modules. 
One has [I, I] = , [[', ['] = and both I and I are (3-isotropic and f2- 
Lagrangian subspaces of q . 

Such an orbit is called holonomy reducible if () acts reducibly on q. 

Proposition 3.2 (J$) Let O be a holonomy reducible symmetric orbit in g. 
We define the map <f> : q = t © I — > O by 

4>(l,l') := Ad{exp{l).exp{l')).o 

Then <f> is a Darboux chart on (0,u> ). Precisely, one has 4>*to = ft. 

We transport the infinitesimal action of g on <p(q) C O to an infinitesimal action 
of g on q via </>. One then gets a homomorphism of Lie algebra 

- X(q) : X - X*. 

Setting 

A A (ic) :=/3(0(a;),A) (xeq,Aeg), 

one obtains the strongly hamiltonian system (cf.Definition [O] ) (q, 57, g, A). The 
main property of the Darboux chart <j> is 

Proposition 3.3 |^/ The Moyal star product on the symplectic vector space 
(q , fl) is Q-covariant. Moreover the Moyal star product in this case satisfies the 
property (B). 

Definition 3.2 Let ^(q) be the space of distributions on q which are tempered 
in the {'-variables w.r.t. the Lebesgue measure dl 1 on I . OnS' 2 (q), we consider 
the partial Fourier transform T : ^(q) — » ^(q) which reads formally as 

(Tu){l, n) := J e- tn{v ' l,) u(l, l')dl' 1 := 

Here q := [© [ i.e. we identify the dual space l' with I by use ofQ. 
We will also adopt the notation u := Tu 

We define the R-isomorphism 

q — * [ : (1, 77) — * z = I + ivn 

where the parameter v is now considered as being real. In q = [©[', we choose 
an 17-symplectic basis {L a ,L' a ; 1 < a < n} where L a £ [ and L' a € l'. On [, we 
define the coordinate system x = tt(x, L' a )L a —: x a L a . 
If h is an element of f), we define its trace as 

spur(fc) :=n([h,L a ],L' a ). 

C 

In this setting, one has the holomorphic constant vector field on I : 
d x » := —(u(L a )i-i(L a )n) (1 < a < n). 
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(P (P (P 

Definition 3.3 Considering I C g , we define, for all A £ g , the polyno- 
mials on I : 



h\{z) :=A c + [A i/C ,z] €tf 
1 a(z) ■= A [C + [A^ c , z] + \[z, [z, A fC ]] e I 



C 



C 

where z E [ and A = A + A ^ + A^ according to the decomposition 

C ,C rC r/C 

0^ = tr © i © r ■ 

Definition 3.4 

T(T^(f)) by 



Definition 3.4 For all A € g^, we define the holomorphic vector field Zj^ E 



where z E I , / S C°°([ , C) anrf where, for all w = U)\ + iw 2 El — I © it, we 
se< w a := «jf + it«2 (1 < a < n). 

In the same way, extending the Killing form (3 and the trace spur <C-linearly to 

C 

f) , we define the complex polynomial of degree 1 : 

r { X ] :=^W(h A Z 1 o) + uspur{h^)). 
Proposition 3.4 |^/ For all A E Q, one has 

-^^(Xa *v u) = T$ M + Z^ .u 

where u is chosen in such a way that the LHS and the RHS make sense (e.g. 
u E S(q) the Schwartz space on q). 

We obtain a similar expression for the right regular * action. 



4 Three different models for ad* 

From Proposition [TJ, straightforward computations yields 
Proposition 4.1 For all X E Q and u E C°°(q) one has 

ad*„(X)u - [{Q{L a ,X r )~n([h x ,L a },.))L' a u + Q(l x ,L' a )L a u 
+ v 2 n([[X r ,L a ],L b ],L' c )L' a L' b L c u. 

Or equivalently, 

^( a d^(X)u)(l,v)={mX t ,v},o)+tv(h x (l))}u(l,r,) (6) 

+ n(i x (i) - ^-[[x^nUlL^iLa^.uii, v ) + n([h x (i), v ],L' a )(L a ) v .u(i, V ), 

where (L v .f )(l, rj) := A \af(l, T) + tL) and 1 < a,b, c < n. 



Let us now turn to the particular case G = SX(2,R). As usual we denote by 
E,F and H the generators of the Lie algebra s[(2,R) satisfying the following 
commutation relations: 

[E, H] = -2E, [F, H] = 2F, [E, F] = H. 

We identify the vector spaces I and [ with R • E and R • F respectively and 
notice that the sub-algebra f) is isomorphic to R • H. 

Consider the Lie algebra representation p v of st(2,R) on the Schwartz space 
5(1 ffi I) given by 

p v {X)f{l,ri) := F{aA* u {X)f), (7) 
According to (||) we have 

p v {X)f{l, v )=m v {X)f{l,r } ) + y v {X)f{l,r ] ), (8) 

where 

m v {X)f := {i(3{[X l >,r,],o)+tv{h x {l)))f, 

y u {X)f := (Qilx^-^ilX^^rjlL'J^+QiihxillvlL'Ml 

By identification of elements I ■ E and 7/ • F with their coefficients I and r) 
respectively we get 

y v {E) = d h y v {H) = 2ldi + 2r,d v , y v {F) = (y 1 ^ - l 2 ^ - 2l V d n . 

Using the identification of [0 I with [ c given by the R-isomorphism (I, rj) i— >• z = 
I + ivr\ we may write the ODE's describing the local flows of y v (X) as follows: 

For E z = 1, for H z = 2z, for F z = -z 2 . 

One recognizes the generating relations for three subgroups of conformal trans- 
formations of the upper half plane H + . 

Under the same assumptions the multiplicative cocycle m u (X) is given by 
mJX) = 2Xu - 2X r Re(z) + -X v Im(z). 

Let us notice that the representation p v "looks like" a representation given by 
the parabolic induction. More precisely, consider the principal series represen- 
tation of the complex Lie group SL(2, C) acting on smooth functions by 

where fj, € C, £ € Z. This action is unitary for Ke(fi) = —2 (sec |D| p. 33). 
The corresponding infinitesimal action of the generator F is given by 

dV^{F)f{z) = (fiRe(z)+i£lm(z))f(z) - z 2 f'(z). 

So we have 
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Theorem 4.2 The star representation ad*,, exponentiates at the Lie group level 
and it is equivalent to the restriction of the principal series representation V^ 1 ' 1 
of the Lie group G c = SL(2,C) to its real non-compact form SL(2,M) for 
/i = —2.1 = v . The intertwining operator is the partial Fourier transform. 

This result can be generalized in the framework of so-called "conformal" or 
Kantor-Koecher-Tits Lie algebras. Indeed, such a Lie algebra is a symmetric 
graded Lie algebra 

= 0-1 ©flo © 0i, 

with [flfejfli] = Qk+i for k ^ I € { — 1,0,1}. Moreover, a Kantor-Koecher-Tits 
algebra has a canonical s[ 2 -triplet (e, /, g), e G g_i, h G g , / S fli defining this 
graduation: 

9k = {X G Q \[h,X]=2kX}. 

The isomorphic go _nl odules g±i are endowed with the additional structure of 
a Jordan algebra. If such an algebra is euclidian (it carries a positive-definite 
bilinear form) then the real connected Lie group G such that g = Lie(G) is the 
Lie group of holomorphic automorphisms of the associated hermitian symmetric 
space of tube type G/K ~ g_i + iC where C denotes the cone of positive 
elements in g_i (see [|[ [n| for more details). Such a "conformal" Lie group is 
generated by translations of linear transformations preserving the cone C 
(the structure group) and the Jordan algebra inversion. 

Using these three families of generators we can show that the star representation 
ad*„ of the Lie group of holomorphic transformations of an hermitian symmetric 
space of tube type is equivalent to the maximal principal series representation 
"Pfa\ of the complex group G c restricted to G for some particular values of 
parameters /i and A. Complete description of representations "P^.a of conformal 
Lie groups in terms of corresponding Jordan algebras is done in |fl9f . 
Unitary irreducible representations of the maximal principal series of the com- 
plex Lie group SX(2,C) become deeply reducible when restricted to the real 
subgroup SL(2, K). We shall discuss the corresponding brunching laws. More- 
over let us notice that an algebraic approach to the problem of restrictions of 
principal series representations of complex semi-simple Lie groups to their real 
forms is given in [ jl7| . 

Back to the SL(2, M)-case, since SU(1, 1) and SL(2, R) are conjugate in SL{2, C) 
we can restrict all our considerations to G — SU(1, 1). Let K = U(l) be the 
maximal compact subgroup of G and let T(G/K,£) denote the set of functions 
/ G r(G) such that f(gk) = Xe(k)f(g) where \e 1S a character of K given by 

Theorem 4.3 For Re [i = —2 the restriction of the principal series representa- 
tion V^' 1 of G to G is equivalent to 

V^\ G ~ L 2 (G/K, I) 8 L 2 {G/K 1 -£). 

For the proof, recall the construction of the parabolic induction for representa- 
tions V fi ' £ (see for example JTo| ) in the compact picture. Let P c be the Borel 

subgroup of G c consisting of the lower triangular matrices p — f ° ^ 1 J , with 

c G C, a G C*. 
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The group G c acts on S = {s G C 2 : ||s|| = 1} and on S = S/ ~ (where s ~ s' 
if there exists A € C, |A| = 1 such that s — As') by 

.9- s = 9(s)/\\g(s)\\. 

The action is transitive and, clearly, the stabilizer of the first basis vector §i is 
the maximal parabolic P c . So S ~ G c /P c . 
If ds denotes the induced measure on 5 1 , then 

^• S ) = ib( S )ir 4 rf S . 

For /ieC and leZ, let us define the character of P c by 

W ^(p) = \a\»(a/\a\y. 

Consider = Indpc w^. Let C°°{S,e) be the space of smooth functions ip 
on S satisfying 

<p(\s) = A-V(s), (9) 
for s G 5, A 6 C, |A| = 1. Then the representation "P^ acts on it by 

7"^(sM«) = V (ff- i -*)iifl- i («)r. 



Let ( , ) be the standard inner product on L 2 (5): (<^i,<p2) = J s l fii(s)p2(s)ds. 
This form is invariant with respect to (7"*' , "P~^'~ 4 ' £ ). Therefore, if Re/i = —2, 
then the representation is unitary. 

The real form G acts on 5 too. Let [ , ] be the sesqui-linear form on C 2 given 
by [z,w] = z±wi — Z1W1. There are 3 orbits for this action given by 

Oi : [s, s] > 2 : [s, s] < 3 : [s, s] = 0. 

We can identify the two first open orbits Oi and O2 with the group G itself by 
g — > g-ei and g — > .g-e2 respectively. Notice also that for k — [ _ _,„<, G if 



x 

one has gk ■ e\ = e l ^g ■ e\ and gk ■ €2 = e~ l ^g ■ &2- The third orbit is closed and 
its measure is zero. 

Consider a smooth, compactly supported function ip on Oi satisfying Relation 
(§). Then the function 

A(<p)=^(s) := ls,s]-^ 2 <p( S ) 

again satisfies Relation (^) and moreover 

TP(g- 1 .s)=V^(gMs)[s,s}-i*/ 2 , (g G G). 

Therefore the correspondence A : ip — + ip is a linear intertwining map from 
P^'^Ig (on Oi) to the left regular action of the group G on the space C%°(G/K,£) 
which is dense in L 2 (G/K,£). Moreover, keeping in mind that rn^frp is a 

G— invariant measure on Oi, we have 

- Re/j-2| 
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So, in the unitary case Re// = —2, the map A becomes a unitary intertwining 
operator. The same consideration for the second open orbit O2 achieves the 
proof. 

To prove the last theorem in a general setting of an arbitrary Hermitian sym- 
metric space G/K of tube type we need an explicit description of all open 
G-orbits on the complex flag manifold G c /P c . The answer to this question is 
given by the Theorem 5.9 in Assume that Z = G c /P c ~ G comp /K is an 
irreducible hermitian symmetric spaces of compact type dual to G/K (G comp de- 
notes a compact real form of the connected, simply connected, complex, simple 
Lie group G ). Fix a Cartan involution corresponding to the decomposition 
= 6 © p, with i = Lie(K). There is a compact Cartan subalgebra t C f of 
0. We denote £(g c ,t c ) the corresponding root system. According to Kostant's 
"cascade construction" one introduces the maximal set of strongly orthogonal 
non-compact positive roots in £(g c ,t c ). This set of cardinality I = rank^Q is 
given by 2 = {£1,.. •>&} where £1 is the maximal root and £ m +i is a maxi- 
mal non-compact positive root orthogonal to {£, . . . , £ m }. Any set of strongly 
orthogonal non-compact positive long roots in X(g c ,t c ) is W(G, T)-conjugate 
to a subset of S, (where W(G,T) denotes the Weyl group of the root system 
S(g c ,t c )). Let z be the base point of the flag manifold Z, and let cr denotes 
the partial Cayley transform associated to a set of non-compact positive roots 

r. 



Theorem 4.4 (\2f\j) The G-orbits on Z are just the orbits 

Or, a = G(c r c 2 A z ) 

where T and A are disjoint subsets ofE,. Two such orbits Or, a ond Or', A' are 
equal if and only if cardinalities \T\ = \T'\ and |A| = |A'|. An orbit Or, a is 
open if and only if T is empty. Any open orbit is maximal-dimensional. 



Therefore, according to theorems 4.3 and 4.4, there exists a finite number of 
irreducible representations (r, V T ) of the maximal compact subgroup K such 
that the restriction of unitary principal series representations of the complex 
hermitian group G c to its non-compact real form decomposes into a direct sum 
of regular representations of G on spaces of the type L 2 (G/K, \t) of L 2 -sections 
of the vector bundle G x k V t . 

Summarizing we can say that the star representation p u with (y = l/£) yields 
an interpolation between the infinitesimal action of g on the space of functions 
on the -ff -reducible coadjoint orbit (more precisely on the image of the Darboux 
chart) on one hand {y = 0) and the infinitesimal action of g on space of functions 
on the hermitian symmetric space of tube type G/K (£ = 0) on the other hand. 

Notice that a similar phenomenon occurs in the context of Berezin quantiza- 
tion. It was recently pointed out by Y.Neretin Jl8[ for the hermitian symmetric 
space X = U(p,q)/U(p) x U{q) (which becomes of tube type for p = q). He 
considered the so-called canonical representations of the group U(p, q) acting in 
Hilbert spaces of compactly supported smooth functions on X endowed with 
scalar products given by positive-definite Berezin kernels B\. Notice that B\ is 
positive-definite precisely for A = A„ negative integer and for A > Const > (A 
belonging to the Wallach set). Taking limits for A„ — > —00 and A — > +00 Neretin 
showed that these unitary representations give rise to an interpolation between 
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regular representations of U (p, q) and its dual compact real form U(p + q) on 
L 2 (U(p,q)/U{p) x U(q)) and L 2 (U(p + q)/U{p) x U{q)) respectively. 

5 The Poincare disc 

The decomposition of L 2 (G/K, £) into a sum of irreducible subspaces is explicitly 
described in ||| ( the more general case of the Plancherel formula for spherical 
functions on a Hermitian Lie group is studied in [|T]). 

Recall the following result by Takahashi (cf (H) 4.20-4.21). Let / belongs to 
L 2 (G/K,£). Then 

1 f 00 

\\M\ 2 = 7T Q,i/2+ir(fe)Ttanh(nT)dT+ V (p - 1 /2)Q. P (h) , (10) 

i<p<W 

where £„ iS denotes the positive definite spherical function on G given by £„ iS (at) = 
(l-X) s 2 F 1 (s+n, s-n, 1, X), where X = tanh 2 (i/2) and a t = 
It acts on L 2 (G/K,£) in sense of distributions. 

Formula ( |Io|) shows that there exists an embedding L 2 (G/K) L 2 (G/K,£) 
intertwining the corresponding regular representations. E.P. van den Ban told us 
an elegant way to realize any of such embedding in terms of Fourier transforms, 
we reproduce this here. For the details, we refer to || and literature cited there. 

Let G be a Hermitian simple Lie group. Write G — KAN for an Iwasawa decom- 
position and consider P — MAN the associated parabolic subgroup. Denote by 
a the Lie algebra of A. 

First, for A € aj, denote by the space of measurable functions / : G — > C 
satisfying the following properties 

(i) f(xman) = a x ~ p f(x), (x G G, (to, a,n) <E M x Ax N), 

(ii) f\ K e L 2 (K). 

This space is isomorphic to the Hilbert space Ti :— L 2 (K/M). The group G 
unitarily acts on H\ \ — Ti by left translations. 

Second, consider an unitary irreducible representation (Vs,S) of K. Consider 
the Hilbert space of sections L 2 (G/K,S) realized as equivariant functions 

L 2 (G/K, S)c{f:G^V s \ f(xk) = 6(h)- 1 f(x), xeG,keK}, (11) 

and define 

H s = Uom K (V S ,H). 

The latter is finite dimensional as isomorphic to Hom(V5, 1) (by Frobenius reci- 
procity). The space Tt s is non trivial if and only if 5 has a non trivial M-fixcd 
vector. 

The operator-valued Fourier transform is then defined as a map T& which asso- 
ciated to / e C^°(G/K,5) the meromorphic function Fgf : a£ — > Hom(7Y 5 ,7i) 
given by 

T s f{X)T= f 7r 1)A (z)T(/(z)) dx, (T e H s ). 

JG 



( cosht sinht 
I sinh t cosh t 
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Observe that the transformation Tg intertwines the left action of G on (G/K, 5) 
with the representation 1 ® ni t \ in (H s )* <g> W. By the Plancherel formula |2|, 
one sees that Tg actually extends to L 2 (G/K, 6) as a partial isometry 

L 2 (G/K,5) -» L 2 (ia* + ,Hom(?< 4 ,W), d/x(A) = rrrW> 

|c(A)| 2 

(as usual c(A) denotes the Harish-Chandra c-function [|5| p. 279, and o* + the 
positive Weyl chamber) intertwining the left action and 

tt s := [ {I s ®m,x)dn{\). 



We denote by Tg its transpose. The transformation T\ is even an isometry as 
it coincides with the usual Fourier transform. 

Note that in the case G = SL(2, R), the space Vg is one-dimensional. Hence the 
Fourier transform Tg takes the simpler form 

W)(A) = / f(x)ir ltX (x)\ HS dx. 



Theorem 5.1 Let 

M* : ia* + -> Hom^,^ 1 ) 

be a measurable function, almost everywhere bounded and such that M*(X) is 

onto for almost every A. Then, the linear map L 2 (G/K) -U L 2 (G/K,6) given 
by 

Tf(\)=T s *(T 1 f(\)oM(\)*), 

is a continuous injective intertwiner. Moreover any such continuous injective 
intertwiner can be realized this way. 

We start by proving the second assertion. If T : L 2 (G/K) — > L 2 (G/K,8) is a 
continuous intertwiner, then the map 

f : L 2 (ia* + ,Rom(n\n),dfi) -> L 2 (ia* + , Rom(H s ,H), dfi) 

defined as T := Tg o T o T^ 1 is a continuous linear map intertwining ir 1 with 
ir s . Two representations 7ri,> and iri t x' (A 7^ A') being irreducible and non- 
equivalent, in vertue of the Schur lemma T is a multiplication operator. Hence, 
there exists a measurable function M : ia* + — > Hom((Tl K )* , (Ti. 6 )*) such that 

(f^)(A) = (M(A)(g)id w )(^(A)) 

viewing Uom{H s , H) as (H 5 )* <g> W. Or equivalently, 

f = ipoM*. (12) 

Note that since H 1 is identified with the one dimensional space H K of _ftT- 
invariant elements in 7i, one has that injectivity of T implies injectivity of 
M(A) almost everywhere. We now prove the almost everywhere boundedncss 
of M. Assume it is not the case, fix e > and consider a subset S of ia* + such 
that < measure(S') < 00 and ||M*(s)|| > ||T||+e. Then one easily finds a unit 
element tp € L 2 {ia*+, Hom(H 1 , H), dfi) such that ||<poM*|| 2 > ||T|| + e (choose 
ip along the characteristic function of S) . The latter contradicts equation ( |l2| ) . 
The rest of Theorem 5.1 being clear, we now may state the result announced in 
the introduction, namely 
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Theorem 5.2 Let £ £ Zo and denote by (H(M. 2 ),pi) the unitary representation 
of g = sl(2, R) obtained by assigning the value j to the parameter v in the formal 
expression of the star representation ad* v of q in the Weyl-Moyal algebra ( cf. 
Proposition 



1, Theorems Jf-.i and ^.jJ. Then, Th 



id \5. 



explicitly describe the infective continuous intertwiners 



L 2 (G/K)^(H(R 2 ) lPe ). 
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